The present invention is directed to a method and system for the creation, enhancement and update of data on a customer database using data from a central database and persistent keys that are linked to each data structure. These persistent keys allow the matching of equivalent data structures between the remote and central databases so that enhancement data from the central database can be overlaid onto the customer database. The persistent keys also make the update of data on the customer database more efficient, since key matching allows the transfer of only the data that is needed to update the customer database. The central database may physically contain all of the enhancement data, or may use a central database manager to transparently link a group of physically remote databases, such that a virtual central database is created containing data from all of the linked databases. The linking feature of the persistent keys allows the central database manager to transparently link the various physically remote databases so that they appear as a unified central database to the customer database operator requesting data enhancement.
The high cost of mass-market advertising has increasingly led many retailers to seek more specific ways to target their advertising to those persons most likely to purchase their products or services. A popular method of directed advertising is to develop a database of prospective customers defined by specific criteria such as age, marital status, geographic region and income. Using such a database, retailers can create a "mailing list" to accurately target mailings to only those persons most likely to respond, thereby gaining the largest return on their advertising expenditures.
A number of companies now offer services such as those described above. These companies (which will be referred to as "data vendors") either maintain or have access to enormous amounts of data pertaining to individuals, businesses and other entities. Drawing on such resources, a data vendor can tailor a mailing list to fit a particular retailer's specific needs. A data vendor's services can be as simple as building a database with the names and addresses of all individuals living in a particular geographic region. Data vendors can also build databases using complex household demographic data as the criteria for inclusion in the database. In addition, data vendors can "enhance" a retailer's in-house database of current customers with additional data. For example, a retailer may want to know the age and marital status of all of the customers on its catalog mailing list. Using such demographic data, retailers can draw conclusions about the type of individuals or businesses that typically use their goods or services and can thus tailor a marketing strategy specifically to individuals or businesses meeting those criteria.
Retailers are not the only consumers of these types of data services. For example, insurance companies may require access to large compilations of such data for underwriting purposes. Credit card issuers also can benefit from compilations of this data in assessing risk and other factors impacting their business. Because of the broad spectrum of entities that may benefit from the extraction of such data, they will be referred to generically as "data customers."
Using prior art methods, the process of database enhancement for such purposes as described above is time consuming and labor intensive. In a typical example of the process, the data customer is first required to build an "extract file" containing an entry for each record in the data customer's database. This extract file is stored on a computer-readable medium, such as magnetic tape, which is then shipped to the data vendor for enhancement.
Since a wide variety of database formats may be used by different data customers, the data vendor must first convert the extract file to the data vendor's internal format for processing. Using this standardized version of the extract file, the data vendor then executes a software application that compares the information in the data customer's database against all of the information that the data vendor maintains or has access to in its various databases. The requested information is typically spread over a number of physically remote databases, which must be individually searched by the data vendor. The data vendor's personnel must create specialized software applications that access each of these physically remote databases to search for the desired data. Additional software is required to compile the data in a central location and "purge" duplicate entries.
The enhancement data is then overlaid onto the data customer's standardized extract file. The resulting file is reformatted into the data customer's database format and written back onto the computer-readable medium for shipment back to the data customer. This process thus requires significant direct involvement by the data vendor's personnel and can in some cases require several weeks from start to finish.
A significant limitation of the current industry method is that each time enhancement data is requested, the entire process must be repeated. Although matching keys are created during the comparison process to match each structure in the data customer's database with the equivalent structure in the data vendor's databases, those keys are temporary and are lost once the process is complete. The keys must be recreated from scratch each time an enhancement task is run, even when the data customer only wants to update existing data. It would be impossible to reuse these keys since they are not unique across all of the data vendor's databases.
A further limitation of the current method is that enhancement data must be pulled from numerous sources using a process that is unique for each physical database accessed. Data vendors do not typically maintain all of the data that they use in one physical database. Instead, the data is spread across many databases, often at remote locations. Thus to enhance a data customer's database, a program must be created that pulls each element of the requested data from the particular remote database where it is located. Each of these databases may reside on a different type of computer, may run on a different operating system, and may store information using a different database software product. The creation of specialized programs to access each of these individual remote databases to find the particular information desired by the data customer consumes the time and energy of data vendor personnel each time a new data enhancement request is submitted.
Another limitation of the current method for database enhancement is that the entire process must be performed when the data customer only requires an update to information already contained in its database. For example, a retailer may desire to update the addresses in its customer database once per month. Most such customers will not have changed their address within this period. However, the current method would require the retailer to order an entirely new set of data that replaces the old set, since there is no means to keep track of which addresses have changed since the retailer's last data update.
Yet another limitation of the current method is that it does not provide the option of near real-time data access. The system is inherently batch mode in operation, since the data for which update or enhancement is required is shipped to the data vendor, who then returns the modified data some time later. There is no means by which a data customer can remotely and automatically establish a linkage between its database and a data vendor's databases so that update or enhancement data can be immediately overlaid onto the data customer's database.
Still another limitation of the current method is the high transactional cost incurred in any update or enhancement, no matter how much data is transferred. The cost of updating or enhancing data reflects the cost both of compiling and maintaining the requested data and of performing the update or enhancement operation itself. Since the cost of the update or enhancement operation is virtually the same no matter how much data is requested, small retailers and other similarly-situated data customers with relatively small databases (say, for example, fewer than 100,000 records) have historically been precluded from the market for data vendors' services. The high transactional costs are incurred because of the extensive human interaction required using the current method for the update or enhancement of any data. The current update and enhancement method is not sufficiently automated to push the associated transaction costs low enough for many smaller entities to afford such services.